Middleware for Mobile Sensing Applications in Urban Environments

Sensor networks represent an attractive tool to observe the physical world. Networks of tiny sensors can be used to detect a fire in a forest, to monitor the level of pollution in a river, or to check on the structural integrity of a bridge. Application-specific deployments of static-sensor networks have been widely investigated. Commonly, these networks involve a centralized data-collection point and no sharing of data outside the organization that owns it. Although this approach can accommodate many application scenarios, it significantly deviates from the pervasive computing vision of ubiquitous sensing where user applications seamlessly access anytime, anywhere data produced by sensors embedded in the surroundings. With the ubiquity and ever-increasing capabilities of mobile devices, urban environments can help give substance to the ubiquitous sensing vision through Urbanets, spontaneously created urban networks. Urbanets consist of mobile multi-sensor devices, such as smart phones and vehicular systems, public sensor networks deployed by municipalities, and individual sensors incorporated in buildings, roads, or daily artifacts. My thesis is that "multi-sensor mobile devices can be successfully programmed to become the underpinning elements of an open, infrastructure-less, distributed sensing platform that can bring sensor data out of their traditional close-loop networks into everyday urban applications". Urbanets can support a variety of services ranging from emergency and surveillance to tourist guidance and entertainment. For instance, cars can be used to provide traffic information services to alert drivers to upcoming traffic jams, and phones to provide shopping recommender services to inform users of special offers at the mall. Urbanets cannot be programmed using traditional distributed computing models, which assume underlying networks with functionally homogeneous nodes, stable configurations, and known delays. Conversely, Urbanets have functionally heterogeneous nodes, volatile configurations, and unknown delays. Instead, solutions developed for sensor networks and mobile ad hoc networks can be leveraged to provide novel architectures that address Urbanet-specific requirements, while providing useful abstractions that hide the network complexity from the programmer. This dissertation presents two middleware architectures that can support mobile sensing applications in Urbanets. Contory offers a declarative programming model that views Urbanets as a distributed sensor database and exposes an SQL-like interface to developers. Context-aware Migratory Services provides a client-server paradigm, where services are capable of migrating to different nodes in the network in order to maintain a continuous and semantically correct interaction with clients. Compared to previous approaches to supporting mobile sensing urban applications, our architectures are entirely distributed and do not assume constant availability of Internet connectivity. In addition, they allow on-demand collection of sensor data with the accuracy and at the frequency required by every application. These architectures have been implemented in Java and tested on smart phones. They have proved successful in supporting several prototype applications and experimental results obtained in ad hoc networks of phones have demonstrated their feasibility with reasonable performance in terms of latency, memory, and energy consumption.Deploying a network of sensors to monitor an environment is a common practice. For example, cameras in museums, supermarkets, or buildings are installed for surveillance purposes. However, while a decade ago, most deployed sensor networks involved a limited number of sensors, wired to a central processing unit, nowadays, the focus is on wireless, distributed, sensing nodes. Sensor technology has greatly advanced in terms of size, power consumption, processing capabilities, and low cost, thus fostering deployments of self-organizing wireless sensor networks over large geographical areas. For example, sensor networks have been used to detect a fire in a forest, to monitor the level of pollution in a river, or to check on the structural integrity of a bridge. Yet, sensor networks are usually perceived as ``something'' remote in the forest or on the battlefield, and regular users do not yet benefit from them. With the ubiquity and ever-increasing capabilities of mobile devices, such as smart phones and computers embedded in cars, urban environments offer the elements necessary to create people-centric mobile sensor networks and support a large variety of so-called sensing applications ranging from emergency and surveillance to tourist guidance and entertainment. For example, near-ubiquitous smart phones with audio and video sensing capabilities and more sensors in the near future can be used to provide shopping recommender services to inform users of special offers at the mall. Sensor-equipped cars can be used to provide traffic information services to alert drivers to upcoming traffic jams. However, urban mobile sensor networks are challenging programming environments due to the dynamism of mobile devices, the resource constraints of battery-powered devices, the software and hardware heterogeneity, and the large number of concurrent applications that they need to support. These requirements hinder the direct adoption of traditional distributed computing platforms developed for static resource-rich networks. This dissertation presents two architectures that can support the development of mobile sensing applications in urban environments. Contory offers a declarative programming model that views the urban network as a distributed sensor database. Context-aware Migratory Services provides a client-server paradigm, where services are capable of migrating to different nodes in the network in order to maintain a continuous interaction with clients. Compared to previous approaches to supporting mobile sensing urban applications, our architectures are entirely distributed and do not assume constant availability of Internet connectivity. These architectures have been implemented in Java and tested on smart phones. They have proved successful in supporting several prototype applications and experimental results obtained in networks of phones have demonstrated their feasibility with reasonable performance in terms of latency, memory, and energy consumption. The proposed architectures offer many opportunities to flexibly and quickly establish customized services that can greatly enhance the users' urban experience. Further steps to fully accomplish people-centric mobile sensing applications will have to address more technical issues as well as social and legal concerns

Appstract: On-The-Fly App Content Semantics With Better Privacy

ABSTRACT Services like Google Now on Tap and Bing Snapp enable new user experiences by understanding the semantics of contents that users consume in their apps. These systems send contents of currently displayed app pages to the cloud to identify relevant entities (e.g., a movie) appearing in the current page and show information related to such entities (e.g., local theaters playing the movie). These new experiences come with privacy concerns as they can send sensitive on-screen data (bank details, medical data, etc.) to the cloud. We propose a novel approach that efficiently extracts app content semantics on the device, without exfiltrating user data. Our solution consists of two phases: an offline, user-agnostic, in-cloud phase that automatically annotates apps' UI elements with stable semantics, and a lightweight on-device phase that assigns semantics to captured app contents on the fly, by matching the annotations. With this automatic approach we annotated 100+ food, dining, and music apps, with accuracy over 80%. Our system implementation for Android and Windows Phone-Appstract-incurs minimal runtime overhead. We built eight use cases on the Appstract framework

Efficient Context-aware Service Discovery in Multi-Protocol Pervasive Environments

International audienceService discovery is a critical functionality of emerging pervasive computing environments. In such environments, service discovery mechanisms need to (i) overcome the heterogeneity of hardware devices, software platforms, and networking infrastructures; and (ii) provide users with an accurate selection of services that meet their current requirements. To address these issues, we have developed the Multi- Protocol Service Discovery and Access (MSDA) middleware platform2, which provides context-aware service discovery and access in pervasive environments. This paper primarily focuses on the design and implementation of the context-awareness support of MSDA. Context-awareness not only provides a more accurate service selection, but also enables a more efficient dissemination of service requests across heterogeneous pervasive environments. We present the design and prototype implementation of MSDA, along with experimental results that demonstrate the advantages derived by introducing context awareness

Contory: A smart phone middleware supporting multiple context provisioning strategies

This paper presents Contory, a middleware specifically deployed to support provisioning of context information on mobile devices such as smart phones. Contory integrates multiple strategies for context provisioning, namely internal sensors-based, external infrastructure-based, and distributed provisioning in ad hoc networks. Applications can query Contory about context items of different types, using a declarative query language which features on-demand, periodic, and event-based context queries. Contory allows applications to utilize different provisioning mechanisms depending on resource availability and presence of external infrastructures. This paper illustrates our approach along with its design and implementation on smart phones. 1

data out of close-loop networks into the center of daily urban life.

With mobile devices becoming ubiquitous, the time is ripe to bring senso